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Abstract. This research reports the devel- 
opment of an online three-tier diagnostic 
instrument for pre-university students 
related to cellular respiration. To date, only 
few studies have been conducted to identify 
students’ alternative conception in the 
topic of cellular respiration. One of the con- 
tributing factors is due to lack of instrument 
developed to measure these alternative 
conceptions. Three-tier tests are used as 
assessment tools for lecturers to determine 
students’ alternative conceptions related 
to cellular respiration and their knowledge 
and understanding about the concepts. The 
research incorporates both qualitative and 
quantitative methods. The instrument was 
developed in five phases: (1) construction 
of items, (2) pilot study, (3) validation of in- 
strument, (4) transforming the instrument 
into an online assessment tool, and (5) the 
administration of the Online Diagnostic 
Tool (ODiT). The Cellular Respiration ODiT 
consists of three tiers: answer and reason- 
ing tiers to measure “content knowledge” 
and “explanatory knowledge” respectively, 
and a third tier that measures the level 

of confidence of pre-university students. 
Analysis of the students’ responses demon- 
strated acceptable reliability and validity 
of the instrument. The research also shows 
that both students and lecturers benefit 
when online diagnostic tests are imple- 
mented appropriately. 
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Introduction 


Students come to biology class with prior knowledge and this knowl- 
edge affects the way learners perceive new scientific information. Some part 
of this knowledge can be different or in conflict with the information that is 
generally accepted by scientists (Mutlu & Sesen, 2016; Taslidere, 2016). The 
alternative information may stand in the way of learning biological concepts 
in a meaningful way and could possibly obstruct the acquisition of new 
knowledge (Lazarowitz & Lieb, 2006). It is now widely acknowledged that 
students’ alternative conceptions in biology can impede their learning and 
meaningful understanding of the subject (Anderson, 1986; Palmer, 2001; 
Taslidere, 2016). In this research, alternative conception refers to students’ 
conceptions that conflict with scientific information. Biology misconceptions 
have been recognized as a major factor affecting students’ understanding 
of science at the secondary school level with many of these misconceptions 
being carried onwards to the university level (Coll & Treagust, 2003). Research 
on students’ understanding of cellular respiration revealed an array of mis- 
conceptions and inaccurate conceptualizations among secondary school 
students (Lazarowitz & Lieb, 2006; Halimin & Shamsudin; 2016), as well as 
pre-service science teachers (Yakisan, 2016; Galvin, Simmie, & O’Grady, 2015). 
For instance, a study conducted by Yusof, Halimin, and Shamsudin (2016) 
investigated the understanding of diffusion, osmosis, and the particulate 
theory of matter and found that university students’ understanding of the 
particulate theory of matter was unsatisfactory. Similarly, Galvin, Simmie, 
and O'Grady (2015) investigated misconceptions about respiration and 
photosynthesis and found an unacceptably high level of misconceptions 
among all pre-service teachers and suggested that diagnostic tests can bea 
useful entry point to a pedagogical cycle for the recognition, reduction, and 
removal of misconceptions. 
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In science education, various types of assessments have been used to identify students’ alternative concep- 
tions (Ayla, 2012). Assessment methods such as open-ended questions, interviews, concept maps, and multiple- 
choice questions have shown both advantages and disadvantages in practical usage (Calik & Ayas, 2005; Goh & 
Chia, 1991; Montfort, Brown, & Findley, 2007; Osborne & Gilbert, 1980; Schmidt, 1997; Thompson & Logue, 2006; 
Tsai & Chou, 2002; Uzuntiryaki & Geban, 2005). According to Treagust (1998) among the assessment techniques, 
multiple-choice diagnostic tests are often more preferable in science classes since they are easy to apply. In- 
structor will be able to evaluate the students’ understanding of the subject based on the result of the diagnostic 
test. However, Treagust (1998) agrees that multiple-choice tests have some limitations in their application, such 
as determining whether a student gives a correct response on a test consciously or just by chance. Interviews 
on the other hand can provide more detailed information about students’ alternative conceptions and their 
understanding of a particular concept. However, to interview a large number of students to generalize their 
alternative conceptions is time consuming (Montfort, Brown, & Findley, 2007; Thompson & Logue, 2006). Since 
the abovementioned techniques have some limitations for practical use in classes, two-tier tests have been pro- 
posed (Treagust, 1995) to strengthen the advantages of multiple-choice tests. The two-tier tests were developed 
to identify students’ alternative conceptions (Tan, Goh, Chia, & Treagust, 2002). A two-tier multiple-choice item 
includes content-based alternatives in the first tier (answer tier or A tier) and relevant principles that justify the 
A-tier response in the second tier (reason tier or R tier) (Caleon & Subramaniam, 2010). The student gives a cor- 
rect response to a question and the second tier requires a reason for the response. Such a format allows one to 
determine any alternative conception the student holds. Treagust (1995) also suggested that teacher can identify 
students’ alternative conceptions of a related topic when diagnostic tests are used either at the beginning or at 
the end of a chosen topic. Thus, based on these alternative conceptions shown by students, the teacher will be 
able to modify the related lesson plan to remedy students’ alternative conceptions. 

Two-tier multiple-choice tests which are relative to typical multiple choices questions (MCQs) can be ef- 
ficiently used to probe students’ alternative conceptions (Caleon & Subramaniam, 2010; Treagust, 1995). Two-tier 
tests measure not only the students’ ability to select correct responses, but also the reasoning behind their choices. 
In spite of this, two-tier multiple-choice tests are not impervious to the key limitation of the multiple-choice 
tests, which is the inability to differentiate mistakes resulting from a lack of knowledge from mistakes due to 
genuine alternative conceptions, and to distinguish correct answers based on guessing from correct answers 
based on genuine understanding (Caleon & Subramaniam, 2010). Thus, an additional tier has been introduced 
to address this limitation. A confidence ratings serve as an additional tier of two-tier multiple-choice tests that 
used to determine students’ confidence level in solving the item of MCQs (Hasan, Bagayoko & Kelley, 1999). The 
three-tier diagnostic tests are considered to be more accurate in eliciting student alternative conceptions, since 
they can detect the lack of knowledge percentages by means of the confidence ratings (Aydin, 2007; Eryilmaz, 
2010; Gurel, Eryilmaz, & McDermott, 2015). This enables teachers to gain valuable information about students’ 
alternative conceptions and provides a foundation for developing a valid and reliable diagnostic assessment tool. 
Stankov and Dolph (2000) refer confidence rating to the “appraisals and judgments by an individual regarding 
the quality or accuracy of his or her own performance” (Caleon, 2010). Confidence ratings have been included 
in testing since the 1930s, mainly in the field of psychology. According to Echternacht (1972), confidence rating 
is use to enhance the amount of information that can be obtained from objective tests. Several studies have 
also shown that confidence rating can be related to academic performance (e.g., Stankov & Crawford, 1997; 
Zakay & Glicksohn, 1992). Unfortunately, research indicates that students tend to be poorly calibrated, that 
is, their confidence level tends to be higher than what is warranted by the accuracy of their response (Lunde- 
berg, Fox, & Punccohar, 1994; Morris, 1990). Lundeberg et al. (1994) define good calibration as expressing high 
confidence when answers are correct and expressing low confidence when answers are wrong. In studies that 
dealt with students’ understanding of texts, confidence ratings were found to increase based on factors such 
as the accessibility of the concept from respondents’ memory (Morris, 1990), level of familiarity or expertise in 
the relevant field (Glenberg et al., 1987), and when self-generated feedback could be obtained from a related 
pretest (Glenberg & Epstein, 1985; Glenberg et al., 1987). 

Knowledge assessment is one of the most important parts of the educational process. Technology in educa- 
tion offers many new opportunities for innovation in educational assessment. The increased use of computers 
in education and the embracement of e-learning lead to computerized assessments, especially web-based as- 
sessment systems. Studies (for e.g. Petrisor et al., 2016) revealed that students have a high level of acceptance 
for the online test as an assessment method. Online assessment are better suited for knowledge assessment 
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and are more objective as opposed to other assessment forms, like pen and paper examination. According to 
Grive, Padgett, and Moffitt (2016), current students no longer hold negative perceptions about using technology 
in teaching and learning. Experiences with technology at the pre-university (undergraduate foundation) level 
can help with the transition to the higher learning institution. The ability to use different technologies prior 
to university serves as a route for students to help them prepare for university life (Oliver et al., 2014). Studies 
show that both students and faculty benefit when online evaluations are implemented appropriately. Online 
assessments can provide the information needed by students and teachers to improve learning through rich 
assessment tasks and potentially powerful scoring, reporting, and real-time feedback mechanisms (Kozma, 
2009; Scalise & Gifford, 2006). Most of the assessments in pre-university biology subjects are paper-based as- 
sessments. Analysis of the results is time consuming and feedback is not obtained instantaneously, especially 
when dealing with a large number of students such as those at the pre-university level. Thus, the development 
of an online three-tier diagnostic test to assess pre-university students’ understanding of cellular respiration has 
been seen as an effective tool to assist teachers in “what” and “how” they teach pre-university students in biology. 
A longitudinal study by Woit and Mason (2003) on the effectiveness of online assessment showed that online 
evaluation can be implemented securely and efficiently, and can result in an increase in students’ motivation 
and performance (Ricketts & Wilks, 2010; Woit & Mason, 2003). Thus, the online diagnostic test will help lectur- 
ers in identifying students’ weak areas of understanding in the topic of cellular respiration and helping them to 
achieve the proficiency needed for entrance to science-based undergraduate courses in universities. 


The Purpose of the Research 


Biology is a unique branch of science (Lin, 2004). Biology knowledge is considered as an extensive, multi- 
faceted and often ill-structured. The domain of biology stretches across great expanses of time and a remark- 
able array of sub-fields. The complexity of this subject, the interconnectedness of knowledge at many different 
levels, and the invisible nature of many key processes make biology a particularly difficult subject to teach and 
to learn (Fisher, Wandersee, & Moody, 2000; Lin, 2004). This research focuses on students’ understanding of cel- 
lular respiration which is typically regarded as a difficult topic for students to learn and for instructors to teach 
since it is a complex biological process that involves integrating multiple concepts (Songer & Mintzes, 1994). 
The process of cell respiration occurs within cell mitochondria whereby energy stored in carbon-containing mol- 
ecules, especially glucose, is converted into high-energy bonds of adenosine triphosphate (ATP), the only usable 
source of energy for many cellular activities. Many students merely memorize the steps in cellular respiration 
that are glycolysis, the citric acid cycle (Krebs cycle), and the electron transport chain. They may not understand 
how these stages relate to one another and the significance of these stages. Assessment of students’ conceptual 
understanding and alternative conception in cellular respiration is important for providing effective instruction. 
Therefore, the purpose of this research was to develop a valid and reliable online three-tier diagnostic test to 
assess students’ understanding of cellular respiration concepts. This research focuses on the following research 
question: Is the Cellular Respiration Online Diagnostic Tool (ODiT) a valid and reliable instrument to determine 
students’ alternative conceptions related to cellular respiration? 


Methodology of the Research 


The research is a mixed-method research that incorporates both qualitative and quantitative methods and 
it involves a sample of Semester 2 pre-university (undergraduate foundation) biology students from a university 
in Malaysia. Data collection process took 18 months to be completed. The instrument was developed in five 
phases: (1) construction of items, (2) pilot study, (3) validation of the Cellular Respiration Diagnostic Instrument, 
(4) transforming the three-tier diagnostic instrument into an online assessment tool, and (5) the administration 
of the Online Diagnostic Tool (ODiT). 
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Sample 


In Malaysia, students have to go through a pre-university preparatory courses such as foundation pro- 
gramme conducted by universities, A-Levels or the national Malaysian Higher School Certificate programme 
after completing high school. The research’s sample came from pre-university students (age=18) that enrolled 
in the foundation of science programme by a university in Malaysia. The programme prepares students with 
comprehensive basic knowledge in Science and Mathematics in order to be competitive to enter critical science 
courses offered by the universities. The students will experience learning in a university learning environment 
and develop their self-esteem. This .program is a one-year course. For the purpose of this research, Semester 2 
students from the biological science program were chosen. The students’ should have acquired the conceptual 
understanding of cellular respiration because it is one of the topics studied in Semester 1. The informed consent 
of the participant was obtained before they participate in this research. Students have been informed about 
the purpose, methods and intended possible uses of the research and their participation was voluntary. Fur- 
thermore, the research did not involve any sensitive issue. Three different groups were selected from the target 
population: (1) the first group completed 15 open-ended questions, which consisted of 28 students (11 males 
and 17 females), (2) the second group was used to validate the instrument, which consisted of 135 students (45 
males and 90 females), and (3) the online instrument was administered to the third group, 526 students (238 
males and 288 females). 


Instrument and Procedures 


The development of ODiT involved five main phases (see Figure 1). The first phase was identifying the con- 
tent boundary of this research and designing the test items. The second phase involved several steps in piloting 
the instrument and the third phase involved the validation of the instrument. The fourth phase was the process 
of transforming the instrument into an online tool and the fifth phase was the administration of ODiT to 526 
pre-university students and getting their feedback on the usability and functionality of the tool via an online 
test. Usability in this research refers to the ease with which an instrument can be administered, interpreted by 
the participant, and scored by the researcher. 


Phase 1: Construction of items 


This phase involved the development of the three-tier MCQs. First, the content boundary of the instrument 
for this research was identified. The questions were developed specifically for the topics of cellular respiration 
in the Semester 1 course on cell biology. These topics were chosen after an extensive literature review, which 
reported that students both nationally and internationally had learning difficulties and held misconceptions 
in these particular topics (Griffard &Wandersee, 2001; Haslam &Treagust, 1987; Mann &Treagust, 1998; Storey, 
1989). The process of respiration plays a very important role in understanding many aspects of living systems. 
Furthermore, these processes are essential in the cycling of matter and energy flow through ecosystems (An- 
derson et al., 1990; Eisen & Stavy, 1988). The majority of students have pre-instructional knowledge or beliefs 
about these concepts and many of them develop only a limited understanding of science concepts following 
instruction (Duit & Treagust, 2003). These students construct sensible and coherent understandings of phenom- 
ena and concepts as seen through their own eyes that do not match the views that are universally accepted by 
the scientific community. Based on the authors’ experience in teaching and assessing students for this topic, 
many misconceptions were observed. 
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The five phases of the online instrument's development. 


Fifteen questions were designed for the research and the questions were mapped according to the Biology 
Curriculum of the pre-university program. Seven questions were constructed to map Course Outcome 1:“students 
should be able to explain the basic concept of cellular respiration”; and eight questions to map Course Outcome 
2:“students should be able to solve problems by applying the basic biological concept of cellular respiration.’The 
first draft of the test items included an answer tier (Tier 1) with open-ended reasoning. The items were then sent 
to two academicians in the related field who were asked to evaluate the items’ effectiveness and suitability. Both 
content experts found the items, 


... to be pretty thorough, which requires that the students know the functional processes taking place within cells 
and the products of such processes. A clear understanding/picture of these processes is required by the students 
as the questions do go into fine detail. Most of the questions would rely on the students remembering/memorizing 
facts. However, a number of the questions do require some analytical thinking or a deeper understanding of the 
facts they have learned. 
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Phase 2: Pilot study 


Once the pilot version of the diagnostic test was developed, it was pilot tested on a sample of 28 students. 
Analysis of the responses in the open-ended questions was carried out to identify any misconceptions among 
the students and then used in the development of the distracters for the second tier of these questions, that is, 
the reasoning tier (Tier 2). Semi-structured interviews were conducted with 10 students who had completed the 
diagnostic test (female=7, male=3) to gain greater depth in understanding their responses in the open-ended 
questions given to them earlier. The students were considered typical and were chosen based on their willingness 
to be interviewed. Typical in this context means that the students were not unusual in any way and they reflected 
the average level of interest (Merriam, 2009; Patton, 2002). The interviews were conducted using questions that 
were similar to those on the developed instrument. Based on the interview findings and the responses in the 
open-ended questions, the distracters for the diagnostic test were identified (Tier 2). The second tier is a reason- 
ing tier, which consisted of an expected reason for the selection made in Tier 1, while the distracters consisted of 
incorrect reasons that came from the students’ responses in the open-ended questions. For example, item 8 in 
this instrument asked about the students’ understanding of aerobic respiration. The term “decarboxylation” refers 
to the reaction that releases CO,, and in aerobic respiration, the “oxidation of glucose,’ which involves the release 
of CO,, occurs in both the link reaction and the Krebs cycle. Glucose is completely broken down at the end of the 
Krebs cycle. From the open-ended responses given by the students, the suitable distracters for this item are: (1) 
ATP is synthesized in the mitochondria. Glucose is broken down and CO, is produced, (2) CO, is one of the products of 
glycolysis, and (3) during the Krebs cycle, CO, will be released from the breakdown of one glucose molecule. The correct 
reasoning is the decarboxylation process occurs only in the link reaction and Krebs cycle. 

A four-point confidence scale (Tier 3) was added below the second tier of each item in the instrument, with 
“1” and “4” corresponding to “Just Guessing” and “Very Confident.’ The instrument was further refined to ensure 
that the reason presented in Tier 2 was consistent with the responses from Tier 1. 


Phase 3: Validation of the three-tier diagnostic instrument 


In this phase, the three-tier diagnostic test was administered for validation to 135 randomly chosen students, 
males and females of the pre-university program. The time allocated for the test was 30 minutes. The students were 
informed that the test was a diagnostic test and the results of the test would not affect their grades, but would 
be used by their lecturers in planning their lessons. An item analysis of the test was carried out to determine the 
difficulty index and the discrimination index, followed by a reliability test (Cronbach's alpha). The analysis helped 
in item selection for the final draft of the test. After this initial evaluation, the questions were altered and deemed 
appropriate for the research. 

The analysis of this test also showed that the instrument is reliable (Cronbach's alpha = 0.864 (> 0.70)). Four 
items out of 15 (27%) were reconstructed due to either inappropriate difficulty level or the discrimination index. 
Eleven items’ (73%) discrimination indexes were greater than 03, thus they were accepted without revision. Ac- 
cording to Lien (1971), a discrimination index greater than 0.3 is considered acceptable. 


Phase 4: Transforming the three-tier diagnostic instrument into an online tool 


The revised diagnostic test from Phase 3 was then converted into an online tool via Google Forms. Google 
Forms was chosen as a platform for this instrument because of its interesting features. The instrument can be 
easily created, the platform offers open access to all users, the students can access the instrument easily to take 
the test and the responses can go directly into a sharable spreadsheet. From the spreadsheet, lecturers can eas- 
ily compute the results and analyze the data (see Figure 2). Google Forms also mobile friendly. ODiT allows the 
students to answer the questions online and allows the researchers to track all the students’ responses instanta- 
neously by referring to their matrix number. ODiT (URL: https://prpum.wordpress.com/) consists of 15 three-tier 
multiple-choice questions, and for each question, the students have to choose a response that best reflects their 
understanding of cellular respiration. 
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Figure 2: Spreadsheet of students’ responses. 


Phase 5: Administration of ODiT 


A different sample of 526 students participated in this phase. ODIT was administered in the university’s 
computer labs. After completing ODIT, the participants also completed an online usability test (see Figure 3) to 
evaluate the usability and friendliness of ODiT. The results of this usability test will be discussed in the results and 
discussion section. 
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Section 2: Computer literacy skills 


Section 3: Evaluation of online diagnostic tool (ODiT) 
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Figure 3: Example of items in usability test. 
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Results of Research 


The Cellular Respiration ODiT was evaluated for its practicality as an assessment tool for lecturers to de- 
termine students’ alternative conceptions related to cellular respiration and their prior knowledge and under- 
standing about the concepts. The responses of 526 students to the 15 three-tier items were analyzed using the 
SPSS statistics software program. 

As mentioned earlier, the instrument was created based on two main constructs: Course Outcome 1 (CO1), 
which assesses the students’ basic understanding of cellular respiration; and Course Outcome 2 (CO2), which 
explores the students’ abilities to solve problems by applying the biological concepts of cellular respiration. The 
total mean scores were 0.605 for the CO1 items (items 1, 4, 5, 6, 7, 9, and 11) and 0.457 for the CO2 items (items 
2, 3,8, 10, 12, 13, 14, and 15). The results show that the students found it more difficult to answer the CO2 items, 
which involved problem-solving skills. From the data analysis, the mean value for the third tier, which measures 
the students’ level of confidence, was 2.235. A mean value nearer to 4 indicates that the confidence level is high. 
For item 7, only 1% of the students gave a correct response to both the content and reasoning parts, even though 
the item involves students’ basic understanding of cellular respiration. Only 10.1% of the 526 students gave a 
correct response to item 14. Examples of the students’ responses to the three-tier test will be discussed in the 
following section. The percentages of the students’ responses are given adjacent to the options. 


Table 1. Item 2 in ODiT (Course Outcome 2). 


Which of the following correctly describes the differences between aerobic respiration and 
anaerobic respiration? 
80% A.* The oxidation of substrates in aerobic respiration is complete, but in anaerobic 
respiration, it is incomplete 
B. The number of oxygen molecules required during aerobic respiration is more than 
that required in anaerobic respiration 
C. The energy released during aerobic respiration is stored in the form of ATP, 
whereas in anaerobic respiration, it is released as heat 
D. The formation of pyruvate only takes place in aerobic respiration 
Which one of the followings is the reason for your answer to the previous question? 
20.4% A. During anaerobic respiration organisms gain less O 2 
71.3% B.* Glucose is completely broken down into ATP through various processes with Q as 
the final electron acceptor but inanaerobic respiration glucose is only broken 
down to pyruvate in glycolysis 
C. In aerobic respiration energy is produced through chemiosmosis, whereas in anaerobic 
respiration energy is converted to heat 
D. _ Since there is no oxygen, pyruvate is not prodwed in anaerobic respiration 
Confidence Rating 
1 2 3 4 
Just Guessing — Unconfident Confident Very Confident 


72.7%* 


Item 2 (see Table 1) assessed students’ understanding of aerobic respiration and anaerobic respiration. 80% 
of the students gave the correct answer A, but a number of them also gave an “incorrect” reason. The majority 
of the students (71.3%) gave both the correct answer and reason, while 20.4% chose reason A, a less accurate 
reason. The students who chose reason A only possessed a basic concept of aerobic and anaerobic respiration 
due to their basic knowledge learned in secondary education, that is, “when there is less or no O,, most organ- 
isms will respire anaerobically” as explained in the following quote, “aerobic respiration requires O,, whereas 
anaerobic respiration does not need O,.” (Student 2, Ln.16-17). Thus, these students’ lack of understanding of 
the concept the “complete oxidation of glucose” prevented them from choosing the correct reason (B). In terms 
of confidence ratings, 72.7% of the students were confident (ratings 3 and 4) in choosing reason A compared 
to 57.2% that were confident in choosing reason B. This is not unexpected since these students had already 
acquired a basic knowledge of the topic in their secondary education. 
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Table 2. Item 9 in ODiT (Course Outcome 1). 


Which of the following statements about the electron transport chain is true? 
Main answer: 
A. — Protons are pump ed out of the mitochondria by the complexes of the electron transport 
chain 
47.4% B.* The proton gradients established during electron transport is a form of potential 
energy 
C. | The movement of protons down a concentration gradient is an endergonic process 
D. ATP synthesis associated with the electron transport chain is an example of substrate 
level phosphorylation 
Which one of the followings is the reason for your answer to the previous question? 
20.3% A. When hydrogen ions are transferred along the electon transport chain, redox 
reaction occurs and energy is released 
42.2% B. Energy flows from higher energy level to lower energy level 
35.9% C.* Protons gradient is created when the concentration of protons in the intermembrane 
space is higher than imtham 
D. The protons pumped out of the mitochondria will then be used to produce ATP 
Confidence Rating: 


| 2 3 th 
Just Guessing — Unconfident Confident Very Confident 
26.1% * 


Item 9 (see Table 2) shows that 47.4% of the students chose the correct answer B, but also gave incorrect 
reasons. Out of this proportion, 35.9% of the students chose the correct reason C, 20.3% chose the incorrect reason 
A, and 42.2% chose the incorrect reason B. The percentage of students that confidently (ratings 3 and 4) answered 
the correct reason C was only 26.1% compared to 46.2% that were confident in choosing incorrect reason A, and 
40.7% were confident in choosing incorrect reason B. 


Table 3. Item 10 in ODiT (Course Outcome 2). 


Chemiosmosis allows redox processes to drive the reaction in which 
Main answer: 
A. — ADPis produced by dephosphorylation of ATP 
B. — Glucose is produced from phosphorylation of ADP 
C. GTP is produced from phosphorylation of ADP 
88.1% D.* ATP is produced by phosphorylation of ADP 
Which one of the followings is the reason for your answer to the previous question? 
58% A. ADP+P —» ATP, energy is absorbed by ADP and P, to form ATP 
B. — Production of ATP from ADP used energy 
32% C.* Energy released during the movement of H through ATP synthase used to bind 
phosphate group and ADP to produce ATP 
D ATP goes through dephosphorylation producing ADP and phosphate 


Confidence Rating: 
1 2 3 4 
Just Guessing  Unconfident Confident Very Confident 
42.1%* 
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In item 10 (see Table 3), the majority of the students (88%) chose the correct answer D, but also gave an 
incorrect reason. The correct reason C was chosen by 32% of the students while incorrect reason A was chosen 
by 58% of the students. For instance, Student 7 admitted that “for this question, | really don’t understand how 
this process (chemiosmosis) can produce the ATP. | just guessed that the answer was‘C”, (Student 7, Ln. 121-122). 
In terms of confidence level, 47.8% of the students were confident (ratings 3 and 4) in choosing the incorrect 
reason A and about 42.1% were confident in choosing the correct reason C. 


Table 4. Item 13 in ODiT (Course Outcome 2). 


Which of the following explains why more ATP molecules are produced from one unit of glucose in 
aerobic respiration compared to fermentation? 
Main answer: 
A. — Enzymes involved in aerob ic respiration are more efficient than those utilized in 
fermentation 
47.4% B.* In aerobic respiration, all the carbon atoms in each unit of glucose are oxidized to 
carbon dioxide 
C. Fermentation produces lactic acid which is toxic to cells 
D. — Fermentation produces alcohol which will inhibit cell metabolism 
Which one of the followings is the reason for your answer to the previous question? 
19% A. In fermentation, lactic acid will be produced which required ATP 
34.5% B.* Energy released from the breakdown df glucose will be transferred to NADH and 
FADH), and then to the electron transport chain 
41.3% C. In aerobic respiration, more enzymes used to activate the process 
D. More CO, produced then more energy released which will be used to produce more ATP 
Confidence Rating: 


1 2 3 4 
Just Guessing — Unconfident Confident Very Confident 
15.6%* 


In item 13 shown in Table 4, 47.4% of the 526 students chose the correct answer B. Out of these, 34.5% 
of the students with the correct answer chose the correct reason B, 41.3% chose incorrect reason C, and 19% 
chose incorrect reason A. Students who chose the correct reason B understood the processes involved in the 
oxidation of glucose and were aware of the significance of these processes in the production of ATP molecules, 
as described by Student 5 in the following excerpt: 


Fermentation is an anaerobic respiration. ... For aerobic respiration, it (glucose) will completely oxidize into carbon 
dioxide, thus ATP will be produced. (Student 5, Ln. 56-57) 


However, more students opted for reason C. The evaluation of the confidence ratings demonstrated that 
66.7% of the students were confident (ratings 3 and 4) in choosing the incorrect answer C, 16.7% were confident 
in choosing incorrect reason A, while only 15.6% were confident in choosing the correct answer B. 

As mentioned earlier, an evaluation of the usability of ODiIT was conducted to understand the students 
thoughts about using the online diagnostic tool. The survey questions also explored the students’ computer 
literacy skills. In this test, 76% of the students agreed that their computer literacy is high. 87% of the students 
used the Web to stay informed in their area of research. 84% of the students admitted that ODiT could be used 
in assessing their understanding of the topic and 70% agreed that ODiT increased their chances of getting better 
grades in this subject. In terms of usability, students gave positive feedback about ODIiT. 82% of the students 
agreed that the instructions in ODiT were clear and understandable and 85% found the instrument to be easy 
to use. In terms of friendliness, 83% of the students found that ODiT was easy to operate. However, some limita- 
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tions were found during the implementation of this instrument. One of the limitations of using Google Forms 
was that the students could not get direct feedback about their results. The feedback on the performance of 
each student goes directly to the lecturers. The lecturers can identify the individual scores of the students and 
are able to share the results with the students in the following classroom instruction. 


Discussion 


The present research examines students’ alternative conception in understanding cellular respiration. 
The mean score in this research indicated that the students’ basic concepts of cellular respiration are still at a 
moderate level. In fact, based on the third tier responses, the students reflected that they were not confident 
in their answers in ODiT. According to Hoskinson, Caballero, and Knight (2013), complex biological problems 
cannot be solved merely by recalling facts from memory. Instead, complex problems require problem solvers 
to engage in a broader range of scientific practices. A higher level of cognitive functioning or expert-like skills 
are necessary to solve the particular problem. The students’ struggle is caused by their inability to apply the 
concepts of biology. Students should be trained to think, describe, and evaluate complex biological concepts. 
They need to think creatively to formulate a problem, find an answer, and evaluate and disseminate it to others 
(Diki, 2013). However, in order for them to be able to evaluate ideas and produce solutions for actual problems, 
they need to understand the basic concepts of cellular respiration. According to researchers in this area (for e.g. 
Bajd, Praprotnik and Matyasek, 2010; Jena, 2014) cellular respiration topic is very difficult to understand. 

From the data analysis, the mean value for the third tier, which measures the students’ level of confidence, 
was 2.235. A mean value nearer to 4 indicates that the confidence level is high. Thus, the result shows that 
overall, the students were not confident when answering the items in ODiT on either tier 1 or tier 2. According 
to Chandrasegaran, Treagust, and Mocerino (2007), the answer to an item is considered to be correct if both 
content and reasoning parts are answered correctly, because this decreases the percentage of students that 
obtain a correct answer by chance (Tsui & Treagust, 2010). The results reflect that the students had difficulties in 
understanding the basic concept of the Krebs cycle and applying the concept of the electron transport chain. 
Some of the findings in this study are in agreement with Bajd, Praprotnik and Matyasek (2010) findings which 
indicated that students do not bring sufficient knowledge about the process of respiration from their upper 
secondary school experience to the pre-university. 

Result from microanalysis of Item 2 implies that most of the students understood the basic concept of 
aerobic and anaerobic respiration, and were able to explain the processes that occur in aerobic respiration 
leading to the “complete oxidation of glucose”. However, some of the students’ still chose a less accurate rea- 
son due to the lack of understanding of the concept the “complete oxidation of glucose”. A comparison of the 
percentage of students who correctly answered both parts of the questions with that of those who correctly 
answered the knowledge part of the questions suggests that many students may have learned facts without 
an adequate understanding of the propositions and concepts involved (Peterson, Treagust & Garnett, 1989). 

Item 9 shows that about half of the participant gave incorrect reasons for the question. In the electron 
transport chain (ETC), H* ions (protons) are pumped from the matrix of mitochondria into the intermembrane 
space of the mitochondria. This causes the concentration of H*ions in the intermembrane space to be higher 
than in the matrix. The difference in the concentration of H* ions creates a “proton gradient,’ which is a form 
of potential energy. The students who chose the incorrect reason A showed that they have only a basic un- 
derstanding of energy production in the electron transport chain, which was acquired during their secondary 
education. The students were careless in choosing incorrect reason B because it refers to the “transfer of elec- 
trons from a high energy level protein carrier to a lower energy level carrier” in the electron transport chain, 
but not the H* ion (protons). 

Item 10 needs the students to understand the mechanics of redox processes which occur in the electron 
transport chain when electrons are transferred from one carrier to another, and energy is released. The released 
energy is then used to produce ATP through chemiosmosis. Chemiosmosis refers to when energy is released 
during electron transfer, and that energy is used to pump H+ ions (protons) from the matrix into the inter- 
membrane space of the mitochondria. The proton gradient created due to a high concentration of protons in 
the intermembrane space causes protons to diffuse back to the matrix through ATP synthase. The potential 
energy released during the movement of protons is used to drive the ATP production reaction. The students 
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that chose the incorrect reason A may only have basic knowledge of chemiosmosis as a process to produce 
ATP; they did not understand the theory behind chemiosmosis and were not able to explain the concept of 
how energy is transferred or transformed in the production of ATP by chemiosmosis. This result is in line with 
research conducted by Jena (2014) who found that the growing misconception status of learners in ATP gen- 
eration from mitochondrial oxidative phosphorylation is high. 

Item 13 requires an understanding of how ATP molecules are produced in aerobic respiration, which 
involves the oxidation of glucose throughout glycolysis, a link reaction, and the Krebs cycle, followed by the 
transfer of energy to nicotinamide adenine dinucleotide (NADH) and Flavin adenine dinucleotide (FADH,, and 
then to the electron transport chain. Result from this item indicates that the students may not understand 
the theory behind the topic, that is, they were not aware of the “energy transfer” concept, whereby energy is 
released by the breaking down (oxidation) of glucose and is transferred to NADH and FADH, and then to the 
electron transport chain. The energy is then used to drive the reaction to produce ATP. Therefore, they could 
not relate “oxidation of glucose” to the role of NADH and FADH.,. 

Based on the overall data analysis and microanalysis of the test items, it appears that the students were 
not able to grasp the basic concept well, thus they were unable to solve the problem-related questions. This 
shows that ODiT can assist lecturers in gauging their students’ understanding of cellular respiration and enable 
them to prepare the necessary actions to address this issue. 


Conclusions 


In light of these results, it can be concluded that the developed online cellular respiration three-tier test 
is a reliable and valid instrument in assessing pre-university students’ conceptual understanding of cellular 
respiration concepts. Despite the limitations, this kind of online instrument has the potential to be used by 
lecturers to determine students’ pre-existing knowledge, even with a large group of students, due to the instan- 
taneous feedback provided to the lecturers. To promote effective and meaningful learning, we need to identify 
the misconceptions and find ways to rectify them. Based on the results of this online three-tier diagnostic 
test, lecturers can design appropriate lessons that will enhance students’ understanding in learning cellular 
respiration. Therefore, the online three-tier diagnostic test developed in this research is helpful in providing 
lecturers with information both on students’ prior knowledge and their alternative conceptions, if any, prior 
to classroom instruction. From the given feedback, lecturers could design and institute relevant measures to 
eliminate their students’ alternative conceptions, for instance, by developing alternative teaching approaches 
that specifically address students’ misconceptions. Further studies could use the ODiT as a tool for assessing 
students’ alternative conceptions of cellular respiration. The biology research literature lacks of three-tier tests 
thus there is need for more studies to develop online three-tier tests in other topics in biology. 
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